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ABSTRACT 

The ULTOR® Passive Pose and Position Engine (P3E) technology, developed by Advanced Optical Systems, Inc 
(AOS), uses real-time image correlation to provide relative position and pose data for spacecraft guidance, navigation, 
and control. Potential data sources include a wide variety of sensors, including visible and infrared cameras. ULTOR® 
P3E has been demonstrated on a number of host processing platforms. 

NASA is integrating ULTOR® P3E into its Relative Navigation System (RNS), which is being developed for the 
upcoming Hubble Space Telescope (HST) Servicing Mission 4 (SM4). During SM4 ULTOR® P3E will perform real- 
time pose and position measurements during both the approach and departure phases of the mission. This paper 
describes the RNS implementation of ULTOR® P3E, and presents results from NASA’s hardware-in-the-loop 
simulation testing against the HST mockup. 
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1. INTRODUCTION 

NASA Goddard Space Flight Center (GSFC) is currently working on a Relative Navigation System (RNS) for the 
upcoming Hubble Space Telescope (HST) Servicing Mission 4 (SM4). Testing for the RNS system was performed at the 
Marshall Space Flight Center (MSFC) Flight Robotics Lab (FRL). Systems such as the RNS are designed to support the 
development of sensor technologies that will be required for the next servicing mission to Hubble which will more than 
likely be fully robotic. In addition, the RNS experiment will help define the sensors needed for relative navigation and 
proximity ops needed for the next generation of NASA vehicles currently under design to support the NASA Vision for 
Space Exploration. 

The technology behind RNS is applicable to more than just SM4. Applications such as automated rendezvous and 
docking (AR&D) require accurate, low-latency relative navigation measurements; sensors that can provide such 
measurements are also useful for crewed missions. Advanced Optical Systems, Inc. (AOS) has developed such sensors, 
one of which -- the ULTOR® Passive Pose and Position Engine (P3E) -- is part of RNS. 

The RNS system consists of cameras, high capacity storage media and a high processing bandwidth hardware platform, 
and will be installed in the shuttle payload bay. The hardware platform will host algorithms including ULTOR® P3E 
that will process image data from the RNS cameras and perform real time measurements of the relative attitude and 
position between the shuttle and the HST. 

AOS developed ULTOR® technology to support Department of Defense missile-based automatic target recognition 
(ATR) needs. ULTOR® for ATR has two key attributes that enabled AOS to extend ULTOR® to pose estimation for 
relative navigation applications. The first attribute is its ability to detect and locate targets in an image. The second is its 
ability to do so at real-time rates and with low latency. These characteristics allowed AOS to develop the ULTOR® P3E 
system to provide full six-degree-of-ffeedom (6DOF) information of a target. The ability of ULTOR® P3E to measure 
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an object’s attitude and position in real time is an enabling technology for Automated Rendezvous and Docking 
(AR&D), a needed capability to support NASA’s Vision for Space Exploration. 

This paper describes the results of two phases of testing performed at the MSFC FRL facility in 2007. During both 
phases, ULTOR® P3E successfully performed pose estimation on mockups of the HST. We present both the results and 
an overview of the preparations for the final flight build of ULTOR® P3E for RNS. We present data from both RNS 
Phase II testing in April 2007 and RNS Phase III testing from November/December 2007. Analysis of the testing at the 
FRL shows that ULTOR® P3E performed well within the specifications required for SM4 with a translation RMS error 
of less than 0.1 meters and a rotational RMS error of less than 1.6 degrees. 


2. ULTOR® P3E SYSTEM FOR RNS 

AOS designed ULTOR® P3E to be both reconfigurable and portable. It therefore can be hosted on a large number of 
hardware platforms. ULTOR® P3E combines a firmware (VHDL) based high speed image processing engine with real- 
time software algorithms that produce 6DOF information from the processed image data. To date, the ULTOR® P3E 
system has been successfully ported to five different processing platforms, including the SpaceCube platform for RNS. 
This flexibility in design allowed us to integrate several RNS external interfaces including: 

• Integration of custom firmware to receive image data from the specific RNS camera 

• Integration of command and control interfaces to other RNS flight subsystems 

• RNS specific software algorithms for the HST target 

During SM4, ULTOR® P3E will identify and locate a number of features in the field of view of the RNS camera(s). 
Using a digital correlation image processing technique, ULTOR® P3E will measure the position of specific HST 
features in the camera field of view. As shown in Figure 1, the feature positions are then input into an N-point 
perspective algorithm. This algorithm processes the measured spatial relationship between the features and compares 
them to the features’ known location as defined by an accurate HST mechanical model. The output is a 6DOF solution 
for the target that will then be translated into quaternion data suitable for relative navigation and control applications. 



Figure 1. Feature based P3E overview. 

For SM4, ULTOR® P3E will not be provided any a priori knowledge of the relative attitude and position between the 
shuttle and the HST. Therefore, ULTOR® P3E system will need to acquire, track and re-acquire the HST autonomously. 
ULTOR® P3E must be able to acquire the HST target from approximately 150m on approach and from approximately 
5m on release. Acquisition will be accomplished by using an onboard library of HST reference filters produced from 
simulated imagery of the HST. ULTOR® P3E compares image data of HST features to a reference filter (or many 
reference filters) in real-time to determine the position of the feature. These filters are based on specific and unique 
features visible on the target. Such features may include handles, bundles of wires, and distinctive structural elements. 1 
The filters are designed to support the full variance in the approach and release trajectories, referred to as trajectory 






dispersion, expected during the SM4 mission. In acquisition mode, ULTOR® P3E will progress through a matrix of 
filters stored in onboard memory until the initial position and attitude of the HST are determined. 

Once the HST has been acquired, ULTOR® P3E transitions into a target tracking mode and progresses through a matrix 
of filters associated with the measured range. For the most part, the filters are arranged in range bins or gates meaning 
that a set of filters is designed to perform over a certain range between the shuttle and the HST. For every range gate, the 
filters associated with the HST features are robust enough to measure the feature’s location at any anticipated angle or 
position. As the Shuttle’s range relative to the HST changes, ULTOR® P3E software will progress to the next 
appropriate range gate. This continues until the HST is no longer in the field of view— near dock and shortly after release, 
the HST will not be in the RNS camera field of view-or until the filters associated with a particular range gate do not 
provide sufficient accuracy in the measurement. In this case, ULTOR® P3E will transition into re-acquire mode. 

For re-acquire, the ULTOR® P3E software will process filters in adjacent range gates to try to re-acquire the HST target. 
If after several tries the HST is not re-acquired, then ULTOR® P3E software will transition to full acquire mode. The 
usual event that would precipitate a re-acquire would be if the HST were to appear significantly different than expected 
or else if one of the RNS cameras stops functioning. 

For RNS, the algorithms we have developed rely on high quality simulated imagery of the HST to produce robust filters. 
The simulated imagery must cover not only the nominal approach and release trajectories but also the expected 
dispersion. AOS has developed an approach to specifying, processing and testing filters generated from simulated 
imagery to verify accuracy, performance and insensitivity to lighting changes. These filters were tested during Phase II 
and Phase III testing at the FRL. 


Table 1: RNS pose accuracy goals 


Target Range 

Lateral 

Range 

Roll 

Yaw/Pitch 

150 meters 

1.0m 

1.0m 

15 deg 

15 deg 

30 meters 

0.3m 

0.5m 

5 deg 

5deg 

5 meters 

0.1m 

0.1m 



5 deg 


The target accuracy goals for the pose algorithms to be flown on SM4 are listed in Table 1. The objective of the testing 
at the FRL was to measure the progress of the pose algorithms in meeting these goals. The next sections describe RNS 
testing and present the results from these tests. 


3. RNS PHASE II TESTING 

In April of 2007, the RNS system was taken to the MSFC FRL lab. A high fidelity mockup of the HST aft bulkhead was 
mounted on the Dynamic Overhead Target Simulator (DOTS) system at the FRL. 



Figure 2: HST mockup mounted on DOTS at the FRL 






DOTS was programmed to move the HST mockup through simulated shuttle trajectories during SM4. In addition to the 
HST mockup, the RNS cameras were mounted to a tower and an adjacent mockup of the shuttle payload bay lights to 
simulate the shuttle payload bay. The FRL also provided a solar simulator that was used to simulate the angle and 
intensity of the sun expected during the SM4 mission. 



Figure 3a: Solar simulator with HST mockup Figure 3b: Payload bay simulator. RNS cameras on 

tower to the right 


Due to limitations in the size of the FRL, only a subset of the full SM4 approach could be simulated. Also due to space 
restrictions, the approach trajectory was divided up into two pieces, labeled “RBar-To-Grapple” and “Grapple-To- 
Berth”. On SM4 the shuttle is properly aligned to the RBar axis at a range of approximately 30m from the HST. The 
grapple position occurs at roughly 10m. In addition to approach, two departure trajectories were planned. 

ULTOR® P3E was configured to acquire and track the HST for every trajectory. Specific filters were created for each 
trajectory to save upload time to the SpaceCube. In flight, the filters will be uploaded from internal memory, though this 
capability was not available for Phase II testing. Quaternion outputs from ULTOR® P3E were sent to other SpaceCube 
processes for logging. In addition, a debug port for ULTOR® P3E was available and several debug parameters were 
displayed in real-time. The debug data was logged for later analysis. 

4. RESULTS OF PHASE II TESTING 

The ULTOR® P3E data was compared to truth data collected during each trajectory at the FRL. An analysis of 
ULTOR® P3E against specification was also performed for every trajectory. In nearly all of the trajectories, ULTOR® 
P3E pose measurements exhibited a constant bias error in range and the pose angles, as shown in Figures 4 and 5. 

We performed a thorough study of the ULTOR® P3E data and the truth data to determine the source of the errors. The 
range error was due to an inaccurate focal length parameter for one of the RNS cameras. Since ULTOR® P3E produced 
pose by measuring the spatial relationships between features, not using the correct focal length produces a scale error in 
range. The source of the roll and yaw error was determined to be misinterpretation of a transform between the ULTOR® 
P3E pose data’s coordinate frame and that of the FRL truth data. Once this was corrected, the image data was re- 
processed through ULTOR® P3E in SpaceCube hardware. These results are shown in Figures 4 and 5. 










During Phase II testing, a NASA Automatic Gain Control (AGC) algorithm was tested in SpaceCube. It was intended to 
continuously modify the RNS camera’s exposure and gain to provide reasonable illumination of the HST target. This 
meant that the AGC would try to keep image pixels from the RNS camera out of saturation and out of total darkness. A 
study of the AGC performance during Phase II testing revealed that the AGC was too reactive to saturated pixels with 
the end result being loss of dynamic range as pixel values were being driven to darker gray levels. Figure 6 contains an 
image from Phase II testing and its associated histogram. 



Figure 6: HST mockup image and histogram from Phase II testing 


The results of Phase II data analysis revealed that ULTOR® P3E needed several improvements including more accurate 
camera parameters, a better understanding of the coordinate transform process and robustness to lighting changes. Table 
2 shows the RMS error for RBar-To-Grapple trajectory data collected during Phase II testing. 

Table 2: ULTOR® P3E RMS error from RBar-To-Grapple trajectory during Phase II tes ting 



RBar-To-Grapple Trajectory 

X Error 

0.08m RMS 

Y Error 

0.10m RMS 

Z Error 

0.51m RMS 

Roll Error 

10 deg RMS 

Pitch Error 

4 deg RMS 

Yaw Error 

15 deg RMS 


5. RNS PHASE IH TESTING 

A thorough characterization of the RNS cameras provided more accurate measurements of key camera parameters such 
as focal length, dynamic range and sensitivity. AOS provided input to the RNS group developing the improved AGC 
algorithm for Phase III and for flight. It was determined the AGC algorithm would be re-designed to maximize the 
information content in the imagery. It was anticipated that the extreme reaction to saturated pixels would be lessened, 
thus producing images with better contrast. 

In addition, we integrated new image pre-processing algorithms into the ULTOR® P3E firmware to improve the 
effective dynamic range of the image data. This included both a contrast stretching algorithm and a common 
photographic edge enhancement technique called un-sharp mask. The contrast stretching algorithm allows for a specified 
number of pixels above a high pixel value threshold and a low pixel value threshold. Then all remaining pixel values are 
re-binned into values between 0 and 256 since the RNS camera image data has 8 bits per pixel. This in effect stretches 








the contrast. As long as the new AGC would not eliminate the dynamic range by suppressing lower grey scale pixel 
values, the ULTOR® P3E pre-processing algorithms design would maximize the contrast and enhance the edges of the 
imagery, which in turn would improve the system’s performance. Both techniques have been successfully implemented 
in earlier versions of the ULTOR® system. 

As in Phase II testing, the HST mockup was mounted to DOTS, the payload bay light simulator was re-constructed and 
the solar simulator was used to provide the lighting anticipated on SM4. Since Phase III testing was the last chance for 
extensive testing before the SM4 flight, it included many more trajectories including a 1/1 0 th scale mockup of the HST to 
simulate long range trajectories (out to 200 meters). 

6. RESULTS OF PHASE III TESTING 

Since Phase III was the most extensive test to be performed prior to flight, ULTOR® P3E software and firmware were 
configured for flight in many respects. The Phase III ULTOR® P3E build included the ability to support auto-loading of 
filters from flash memory using an external processor. We also developed a single compressed filter set to cover the full 
approach trajectories. In Phase II testing, we had a single uncompressed filter set for each trajectory. The filters for 
Phase III testing were designed to perform over a wider dispersion. Phase III testing included several different 
trajectories including: 

• RBar-To-Grapple 

• Grapple-To-Berth 

• 1/10 Scale Initial Approach 

• 1/10 Scale Near RBar Approach 

• Full Scale Near RBar Approach 

• HST Release 

• Simulated “SM5” approach trajectories 
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Figure 7: ULTOR® P3E results versus truth for one run during Phase III testing. HST leaves field of view after 450 sec. 




The contrast stretching and un-sharp mask algorithms consistently provided quality imagery to the ULTOR® P3E 
processing imagery and the accuracy of the system improved over Phase II. In addition, the new AGC algorithm 
provided images with much improved range as compared to those in Phase II. Several of the trajectories were run 
multiple times to tweak the AGC parameters as well as to compare runs with and without the solar simulator. ULTOR® 
P3E was able to successfully perform pose estimation regardless of the lighting conditions. 
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Figure 8: ULTOR® P3E position and angular error RBar-To-Grapple Trajectory. HST leaves field of view after 450 sec. 

Figure 7 shows the data from a Phase III RBar-To-Grapple trajectory run. The plots compare ULTOR® P3E 
performance to FRL truth data. A pose quality factor was implemented in ULTOR® P3E software for Phase III testing. 
It was based on a number of parameters including the number of identified features, the “goodness” of correlation for 
each feature and the computational time required to derive a pose solution. Only data with a pose quality factor of 60 
(out of 256) or higher is shown. 

Table 3: ULTOR® P3E RMS error from RBar-To-Grapple trajectory during Phase III te sting 



RBar-To-Grapple Trajectory 

X Error 

0.08m RMS 

Y Error 

0.08m RMS 

Z Error 

0.09m RMS 

Roll Error 

.42 deg RMS 

Pitch Error 

1 .23deg RMS 

Yaw Error 

1.53 deg RMS 


The error plot for ULTOR® P3E versus truth for the RBar-To-Grapple trajectory is shown in Figure 8. In both Figures 7 
and 8, the measurement becomes less accurate around 450 seconds as the HST mockup leaves the field of view of the 
RNS camera. The plots also show that the “bias” errors due to focal length and transform errors were corrected in Phase 
III testing. Table 3 shows the RMS error for the RBar-To-Grapple trajectory. This data shows that ULTOR® P3E 




performance has improved significantly since Phase II testing. The angular RMS error was reduced by nearly an order 
of magnitude. The Z (range) RMS error was reduced to just 20% of its Phase II value. 

7. ULTOR® P3E PATH TO FLIGHT 

With the completion of Phase III testing, focus was placed on making final changes to ULTOR® P3E to support the 
SM4 flight. These improvements will include filters that include additional features on the HST that were not present on 
the HST mockup during Phase II and Phase III testing. 

New training imagery will be produced and processed into filter data to support the full dispersions for SM4 which are 
shown in Table 4. These wide dispersions were not required for Phase II or Phase III testing due to FRL and DOTS 
limitations of movement. In addition to new filters, several algorithm changes will be made to support the wider 
dispersion. 


Table 4: Anticipated angular dispersion of trajectory during SM4 



Pre-Grapple 

Post Grapple 

Departure 

Roll 

+- 20 degrees 

+/- 5 degrees 

+/- 3 degrees 

Pitch 

+/- 20 degrees 

+/- 5 degrees 

+/- 3 degrees 


A new acquisition algorithm will be developed to minimize the amount of time needed to identify the position and 
attitude of the HST target. With the increased number of filters due to the wide angular dispersion, an algorithm other 
than “brute force” will be required to acquire. A vector based search algorithm will be developed and integrated into 
ULTOR® P3E software that will reduce the time for acquisition. 

8. CONCLUSIONS 

ULTOR® P3E has consistently shown that it can provide real-time, low latency pose measurements that meet the system 
requirements for a number of Department Of Defense and space related hardware platforms. It has been tested in 
multiple hardware-in the-loop facilities including the FRL for RNS. The ability to perform pose estimation using passive 
optical sensors will be needed to support NASA’s Vision for Space Exploration. 

During RNS Phase II and Phase III testing, AOS has adapted ULTOR® P3E to meet the unique requirements of the 
SM4 mission and continuously improved its performance throughout the program. The ULTOR® P3E system has 
demonstrated flexibility in supporting the interfaces required for integration into the RNS hardware. Phase II and Phase 
III testing has confirmed that ULTOR® P3E can meet or exceed the specification for pose estimation on SM4. 

With the completion of Phase III testing, focus will be placed on the flight ULTOR® P3E build that will provide for a 
successful SM4 and RNS mission in late 2008. The techniques and algorithms developed during RNS will position 
ULTOR® P3E to support the next generation of space vehicles, such as the ORION, with key relative navigation sensor 
technology. 
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